Quantized Neural Networks (QNNs), which use low bitwidth numbers forrepresenting parameters and performing computations, have been proposed toreduce the computation complexity, storage size and memory usage. In QNNs,parameters and activations are uniformly quantized, such that themultiplications and additions can be accelerated by bitwise operations.However, distributions of parameters in Neural Networks are often imbalanced,such that the uniform quantization determined from extremal values may underutilize available bitwidth. In this paper, we propose a novel quantizationmethod that can ensure the balance of distributions of quantized values. Ourmethod first recursively partitions the parameters by percentiles into balancedbins, and then applies uniform quantization. We also introduce computationallycheaper approximations of percentiles to reduce the computation overheadintroduced. Overall, our method improves the prediction accuracies of QNNswithout introducing extra computation during inference, has negligible impacton training speed, and is applicable to both Convolutional Neural Networks andRecurrent Neural Networks. Experiments on standard datasets including ImageNetand Penn Treebank confirm the effectiveness of our method. On ImageNet, thetop-5 error rate of our 4-bit quantized GoogLeNet model is 12.7\%, which issuperior to the state-of-the-arts of QNNs.
展开▼
